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feJESTCP 
Transforming  Munitions  Response 


MMRP 

1000s  of  sites 
Millions  of  acres 
Billions  of  dollars 
Limited  resources 
Decades  to  clean  up 


Costs  are  dominated  by 
digging  scrap 

•  Often  <1%  are  UXO 

•  Example:  Camp  Butner,  NC 

•  146  UXO  out  of  >500,000  digs 

•  Only  0.03%  are  UXO! 
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Site 

Assessment 


Survey  and 
Mapping 


Vegetation 

Removal 


Scrap  Metal  UXO  Removal 
Removal  &  Disposal 


(Average  From 
19  Response  Actions) 


•  The  current  annual  cleanup  effort  is  on  the  order  of  a  few 
percent  of  the  projected  total  cost 

•  To  make  real  progress  on  this  problem,  we  need  a  better 
approach 

•  Classification  offers  the  chance  to  divide  anomalies  into 
those  caused  by  targets-of-interest  and  those  caused  by 
other  things 
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•  A  principled,  data-based  approach  to  classify  targets  as 
either  “non-hazardous”  or  “targets  of  interest” 

1.  Data  Acquisition  2.  Feature  estimation  3.  Classification 

n  *  z  w  s* 

*0  v  &  I?  •• 

«<  *}  □  t.  •• 

••  jl  T  * 


Feature  vector 
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•  Technology  Development  Objectives 

♦  Advanced  Processing  for  Commercial  Sensors 

♦  Purpose-built  Sensors  for  Classification 

•  Testing  and  Demonstration  Timeline 

♦  Late  1990s  -  careful  data  collection  with  commercial  sensors,  mostly 
improves  detection 

♦  2000  -  demonstrations  of  crude  classification  ability  with  commercial 
sensors  on  controlled  sites 

♦  2000-2001  -  SEED  project  to  design  sensor  for  UXO  classification 
(BUD) 

♦  2005  -  build  and  demonstrate  BUD  on  controlled  site  -  successful 
demo  of  developmental  system 

♦  2007-present-  develop  and  demonstrate  transitionable  systems, 
including  HW,  analysis  procedures,  SOPs 
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Live  Site  UXO  Classification  Demonstrations 


•  Goal:  Validate  Classification  Technologies 

♦  Establish  performance  capability  as  function  of  site  conditions 

♦  Establish  operational  procedures  and  costs 

♦  Train  government  and  contractor  community 

♦  Gain  regulatory  acceptance 


•  Completed 

♦  Former  Camp  Sibert,  AL  -  simple  site,  single  munitions  type 

♦  Former  Camp  San  Luis  Obispo,  CA  -  more  difficult  terrain,  mix 
of  munitions,  medium  to  large  size 

♦  Former  Camp  Butner,  NC  -  mix  of  small  and  large  munitions  - 
37  mm,  105  mm,  155  mm 

•  Ongoing 

♦  Mare  Island  Naval  Shipyard,  CA  -  industrial  site 

♦  Pole  Mountain,  WY  -  case  study  in  implementation 

♦  Former  Camp  Beale,  CA  -  trees,  restricted  access 

♦  Fort  Sill,  OK  -  using  classification  in  an  Rl 

•  Planned  -  additional  demonstrations  in  FY12-14 
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•  Seed  the  site  with  inert  munitions 

•  Collect  geophysical  survey  data 

•  Select  potential  targets  -  science-based  threshold  for 
anomaly  picking 

•  Collect  “cued”  geophysical  data 

•  Dig  every  anomaly  that  exceeds  threshold  for  validation 
-  close  hold 

•  Pass  anomaly  locations  to  analysts  -  one  answer  for 
every  location 

•  Score  blind  test  against  ground  truth  -  seeds  and  dug 
targets 
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•  Central  California  Coast 

•  Munitions  types: 

♦  60-mm,  81  -mm  and  4.2-in 
mortar 

♦  2.36-in  rocket 

•  Low  anomaly  density 

♦  100-200  per  acre 

•  Benign  terrain  and 
geology 
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Camp  Butner 


•  North  Carolina 

•  Munitions  types: 

♦  105-mm  and  155-mm 
projectiles 

♦  37-mm  rounds 

•  High  anomaly  density 

♦  Locally,  up  to  800  per  acre 

•  Benign  terrain  and 
geology 
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Why  Should  You  Consider  Classification? 


•  Current  practice  does  not  remove  all  detectable  metal 
from  the  site 

•  Current  field  methods  involve  implicit  discrimination 

♦  Mag  &  Flag  -  instrument  sensitivity  setting  and  human 
interpretation 

♦  Digital  Geophysics  -  threshold  selection;  other  anomaly 
selection  criteria 


Classification  offers  a  better  understood  and  documented  product 

Rigorous 

Tied  to  Targets  of  Interest 
Reviewable  at  Every  Step 
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What  You  Should  Get  From  This  Course 


•  Classification  has  been  successfully  demonstrated  on 
several  sites 

•  It  is  likely  to  be  proposed  for  use  while  demonstrations 
are  ongoing 

•  You  should  be  able  to 

♦  Determine  if  a  proposed  classification  method  is  valid 

■  Method  and  workflow  consistent  with  what  has  been  demonstrated 

■  Site  conditions  suitable 

♦  Know  what  questions  to  ask  and  what  deliverables  to  expect 

♦  Understand  quality  control  considerations 
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Classification  Basics 

Tom  Bell 


4»estcp 

V 


•  Stages  in  the  classification  process 

•  Sensors 

♦  Electromagnetic  induction  (EMI)  fundamentals 

♦  Conventional  vs.  classification-specific  sensor  technology 

•  EMI  Response  Features 

♦  Data  Inversion 

♦  Principal  axis  polarizability 

•  Classification 

♦  Statistical  classifiers 

♦  Library/template  matching 
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Stages  in  the  Classification  Process 


1.  Measure  target  responses 
with  suitable  sensor 
-  Classification-specific  EMI 


log  T 


2.  Extract  target  features 
from  the  measured 
responses 


2 

“  0.03 
|  0.02 
0.01 
0.00 

0  12  3  4 

Size  Parameter 


-  Data  Inversion  3  classify  targets  based  on 

-  Target  polarizabilities  the  features 

-  Statistical  classifiers 

-  Library  matching 
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•  Electromagnetic  Induction  (EMI)  sensors  measure  the 
response  of  nearby  metal  objects  to  magnetic  fields 
created  by  currents  running  through  a  loop  of  wire 


Induced  field  receive  coils 


Primary  field  transmit  coil 
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Electromagnetic  Induction  (EMI) 


•  Magnetic  field  is  produced 
by  running  current  through 
the  transmit  coil 
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Primary  Field 
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V 

(EMI) 


•  Magnetic  field  is  produced 
by  running  current  through 
the  transmit  coil 

•  Current  eddies  spring  up 
in  nearby  metal  objects 
when  the  field  is  cut  off, 
then  quickly  (~10  msec) 
decay 
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Electromagnetic  Induction 
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Electromagnetic  Induction  (EMI) 


•  Magnetic  field  is  produced 
by  running  current  through 
the  transmit  coil 

•  Current  eddies  spring  up 
in  nearby  metal  objects 
when  the  field  is  cut  off, 
then  quickly  (~10  msec) 
decay 

•  Sensor  measures  voltage 
induced  in  receive  coil  by 
decaying  magnetic  field 
from  eddy  currents 
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EMI  Signals 


•  EMI  response  signal  determined  by  target  properties 

♦  Size  and  Shape 

♦  Material  type  and  thickness 


•  Muddled  by  response  variation  with  target  location  and 
orientation  relative  to  primary  field 
♦  Signal  strength  varies  as  sixth  power  of  range 
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EMI  Response  -  Dipole  Model 

Eddy  current  effects  can  be  represented  by  an  induced 
magnetic  moment  (M) 

♦  Strength  decays  with  time  as  eddy  currents  die  out,  decay 
trajectory  is  determined  by  physical  properties  of  target 

♦  Orientation  determined  by  direction  of  primary  field  (H0)  relative 
to  target’s  principal  axes 
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Principal  Axes  &  Polarizabilities 

•  EMI  response  is  decomposed  into  components  along 
three  orthogonal  principal  axis  directions 

♦  Principal  axis  directions 
correspond  to  fundamental 
excitation  modes  of  target 

♦  Magnetic  polarizabilities  are 
specific  responses  to  unit 
excitation  along  each  of 
target’s  principal  axis 

•  Principal  axis  polarizabilities 
completely  describe  EM 
response  of  target 
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Polarizability  Properties 


Generally  simple  form 

♦  Power  laws  trailing  off  to 
exponential  decay 

Decay  rate  depends  mainly 
on  material  thickness 

Relative  strengths  for 
different  principal  axes 
indicative  of  target  shape 

♦  2  secondary  polarizabilities 
equal  for  axisymmetric  objects 

Magnitude  scales  with  target 
size  (volume) 
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•  Generally  simple  form 

♦  Power  laws  trailing  off  to 
exponential  decay 

•  Decay  rate  depends  mainly 
on  material  thickness 

•  Relative  strengths  for 
different  principal  axes 
indicative  of  target  shape 

♦  2  secondary  polarizabilities 
equal  for  axisymmetric  objects 
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size  (volume) 
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Conventional  EMI  Technology 


•  EM61  -  Industry  standard  for  UXO 
detection  work 

•  Not  usually  good  for  classification 
(some  simple  case  exceptions) 

♦  Coarse  measurement  of  eddy  current 
decay  (four  time  windows  or  gates) 

♦  Point  response  measured  at  series  of 
locations  must  be  combined  to  fully 
interrogate  target 

♦  Small  sensor  location  errors  (~1  cm) 
compromise  ability  to  estimate 
polarizability 
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•  Specifically  designed  for  classification 

♦  Measure  complete  eddy  current  decay  signal 

♦  Employ  fixed  arrays  for  precise  sensor  positioning,  and 

♦  Multi-axis  transmit/receive  coils  for  complete  target  illumination 
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Classification-Specific  Sensors 

•  Multi-axis  coils  excite  target  (or  measure  its  response) 
in  different  directions  to  fully  sample  polarizability 


•  Multiple  receivers  enable  precise  determination  of 
target  location 

♦  Crucial  for  accurate  calculation  of  polarizability 


15 
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Stages  in  the  Classification  Process 


1 .  Measure  target  responses 
with  suitable  sensor 
-  Classification-specific  EMI 


log  T 


2.  Extract  target  features 
from  the  measured 
responses 

-  Data  Inversion 

-  Target  polarizabilities 
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Feature  Extraction  (Inversion) 


•  Iterative  search  determines  target  parameters  (location, 
orientation,  polarizability)  for  best  match  between 
EMI  response  model  and  measured  response 


Principal  Axis  Polarizability 


Principal  axis  polarizability  curves  completely  specify 
target’s  EMI  response  characteristics 

♦  Independent  of  sensor/geometry 

♦  Contain  all  information  useful  for  classification 


Time  (msec) 
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•  EMI  signatures  for  weak 
signals  (<10mV)  corrupted 
by  noise 


•  Overlapping  signals  from 
nearby  targets 
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Multiple/Overlapping  Targets 


b) 


Interfering  signals  from  multiple 
targets  in  field  of  view 

a)  Edit  out  competing  signals  from  small 
surface  clutter 

Use  multi-target  solvers  for  larger 
overlapping  targets 
Determine  where  contributions  to  signal 
are  coming  from 

Perform  multi-dipole  inversion  centered 
on  potential  target  locations  (few  targets) 

Preemptive  dig  if  anomaly  has  too 
many  interfering  targets 


C) 


Beale  147_sean2 

Metal  Mapper 

Number  of  targets:  2 

Fit  coh  using  point  targets  &  betas  sf 

Best  coh  using  target  clouds: 


Implementing  Classification  -  Classification  Basics 


Classification  Basics 


10 


Implementing  Classification  on  a  Munitions  Response  Site 


December  1, 2011 


Stages  in  the  Classification  Process 


1.  Measure  target  responses 
with  suitable  sensor 
-  Classification-specific  EMI 


togt 


2.  Extract  target  features 
from  the  measured 
responses 
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-  Data  Inversion  3.  Classify  targets  based  on 

-  Target  polarizabilities  the  featu  res 

-  Statistical  classifiers 

-  Library  matching 
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Simple  Response  Features 

•  Simple  properties  of  principal  axis  polarizabilities  can 
distinguish  between  some  types  of  objects 

•  Large  vs.  small 

♦  Overall  strength  of  response  at 
early  time  (-0.1  msec)  scales 
with  size  of  object 

•  Thin-  vs.  thick-walled 

♦  Strength  of  response  at  later 
time  (-2  msec)  relative  to  early 
time  scales  with  thickness  of 
material 
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Simple  Feature  Spaces 


Simple  two  dimensional  “feature  spaces”  like  size/decay 
can  be  useful  for  visualizing  data  and  identifying  clusters 
of  similar  objects 

♦  “Feature  vector”  is  a  point 
(size,  decay)  in  the  feature 
space 


Size  =  log 
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Simple  Feature  Spaces 


Simple  two  dimensional  “feature  spaces”  like  size/decay 
can  be  useful  for  visualizing  data  and  identifying  clusters 
of  similar  objects 

♦  “Feature  vector”  is  a  point 
(size,  decay)  in  the  feature 
space  -  e.g.  (2,  0.03) 
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Simple  Feature  Spaces 


Simple  two  dimensional  “feature  spaces”  like  size/decay 
can  be  useful  for  visualizing  data  and  identifying  clusters 
of  similar  objects 


♦  “Feature  vector”  is  a  point 
(size,  decay)  in  the  feature 
space 

•  Limited  classification 
power  relative  to  complete 
set  of  polarizability  curves 
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Classification  Techniques 


•  Statistical  Classifiers 

♦  Input  features  include  all  3axesxNgates  polarizabilities 

♦  Machine  learning  -  support  vector  machines,  neural  nets,  etc. 

♦  Trained  on  prior  target  information  and  labeled  training  data 

•  Library  matching 

♦  Asks  what  an  unknown  target  “looks  like”  in  EMI  sense 

♦  Compares  polarizability  against  bank  of  signatures  for  expected 
munitions  and  other  training  objects 

•  Both  approaches  are  based  on  signal  matching 

♦  Statistical  classifiers  create  their  own  library 

♦  Both  can  have  problems  with  unexpected  munitions  types 

Implementing  Classification  -  Classification  Basics 


26 


Classification  Basics 


13 


Implementing  Classification  on  a  Munitions  Response  Site 


December  1, 2011 


Statistical  Classification  Approach 


•  Locate  expected  munitions 
item  signatures  in  feature 
space 
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Statistical  Classification  Approach 


Locate  expected  munitions 
item  signatures  in  feature 
space 

Sample  feature  space  (dig 
targets  &  identify/label) 

♦  Regions  around  munitions 

♦  Target  clusters 


MetalMapper  (Sky  Research) 
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Statistical  Classification  Approach 


•  Locate  expected  munitions 
item  signatures  in  feature 
space 

•  Sample  feature  space  (dig 
targets  &  identify/label) 

♦  Regions  around  munitions 

♦  Target  clusters 

•  Train  classifier  with  labeled 
features 

♦  Set  decision  boundary  to 
exclude  high  confidence 
clutter 
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MetalMapper  (Sky  Research) 
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Template/Library  Matching 


Matched  filters  applied  to  principal  axis  polarizabilities 

♦  What  does  target  “look  like”? 

♦  Compare  against  bank  of  signatures  for  expected  munitions  and 
other  training  objects 


0  1  2  3  4  5 

Size  Parameter 
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•  How  much  alike  do  a  pair  of  EMI  signatures  look? 

•  37mm  and  75mm  projectile  look  somewhat  similar  but 
different  size 
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Signature  Matching 


How  much  alike  do  a  pair  of  EMI  signatures  look? 
Horseshoe  looks  very  different  than  37mm 
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•  How  much  alike  do  a  pair  of  EMI  signatures  look? 

•  Frag  looks  somewhat  different  than  37mm 
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Signature  Variability 


•  EMI  signatures  of  nominally  identical  items  can  differ 

♦  Different  subtypes,  damage,  inversion  errors  due  to  noise 

•  Matching  procedures  must  tolerate  some  variability 
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•  Data/inversion  problems 
can  cause  classification 
ambiguity 

♦  e.g.,  target  PM  1-350  (frag) 
has  poorly  resolved  P3 
polarizability,  but  P1  and  P2 
are  similar  to  PM1-567 
seed  pipe  section  (ISO) 

♦  “Cannot  decide” 

•  Presumptive  munitions  or 
other  target  of  interest 

♦  Generally  must  dig 
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Unexpected  Munitions 


Classify  munitions-sized  targets 
with  axial  symmetry  (P3~P2<Pi) 

♦  e.g.  Stokes  mortar  (Camp  SLO) 
Classify  groups  of  targets  with 
similar  polarizability 

♦  Sample  dig  to  identify  10'r 


♦  e.g.  fuze  parts  (Camp  Beale) 


1.0 
time  (ms) 
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fuze  cluster 
Camp  Beale 
(6  of  25) 
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•  Ranked  anomaly  list 

♦  Likely  munitions 

♦  Likely  clutter 

♦  Can’t  tell 

•  “Stop  dig”  threshold 

♦  Set  to  exclude  only 
high  confidence 
clutter 

♦  Dig  all  uncertain 
targets  (and  likely 
munitions) 
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•  Modern  sensors  accurately  measure  EMI  response 
information  needed  for  reliable  classification 

•  Analysis  procedures  remove  sensor/geometry  effects  to 
extract  target’s  intrinsic  EMI  signature 

♦  Depends  only  on  size,  shape  and  material  properties  of  target 

•  Statistical  and  library-based  classifiers  can  reliably 
distinguish  between  munitions  and  clutter  items 

♦  Match  unknown  targets  with  other  objects  with  similar  EMI 
signatures  (i.e.,  things  they  “look  like”) 

♦  Presumptive  UXO  (dig)  if  EMI  “vision”  is  fuzzy  or  obscured 
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Workflow  and  Quality  Control  Products 

Bryan  Harre 
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•  Identify  important  questions  to  ask  on  a  project  involving 
classification  for  different  phases  of  the  work 

•  Provide  example  data  products  for  each  phase  of  the 
work 

•  Describe  the  quality  control  (QC)  considerations  for  each 
task 

•  Discuss  where  QC  considerations  are  documented  in 
MEC  UFP-QAPP 
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MR  Project  Work  Elements  that 
Classification  Significantly  Changes 


GIS  setup 

Document  management  and 
control 

Subcontracting 

Technical  and  operational 
approach 

Work  Plan  preparation  and 
approval 

Site  prep  and  mobilization 
Site  survey/grid  layout 
Vegetation  removal 
Surface  removal 
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Geophysical  System 
Verification  (GSV) 

Geophysical  survey,  data 
collection,  and  processing 

Anomaly  reacquisition  and 
investigation 

MEC/MPPEH  management 

Demobilization 
Final  report 
Archiving 
Project  closeout 
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Site  Characterization 
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Is  Classification  Applicable  at  This  Site 


•  What  are  the  targets  of  interest  at  this  site? 

♦  Historical  research 

♦  Recovered  munitions  and  fragments/scrap 

♦  Depth  and  density 


Photo  courtesy  of  Estrella  Warbirds  Museum. 
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Is  Classification  Applicable  at  This  Site? 


What  is  the  appropriate  threshold  for  detection? 

♦  Clutter  environment  and  geology  affect  detection  threshold  and 
the  ability  to  classify 

♦  If  project  objectives  require  “picking  into  the  noise,”  you  need  a 
new  plan. 

10 


§  5  '  ' 
!  0 
-5  i 


co 


l__; _ _ _ j 

10 


o>  0 

55  -5 


20 

Distance  (m) 


\  a  A  a  r\l  \i  V  a 

20 

Distance  (m) 


Implementing  Classification  -  Workflow  &  QC  Products 


Workflow  and  Quality  Control  Products 


4 


Implementing  Classification  on  a  Munitions  Response  Project 


December  1,  201 1 


Classification  Planning 


ESTCP 


•  What  classification  sensor  is  appropriate  for  the  site? 

♦  Benign  terrain  -  MetalMapper 

♦  Rougher  terrain  with  steeper  slopes  and  trees  -  TEMTADS  2x2, 
MPV,  handheld  BUD 


MetalMapper.  Photo  courtesy  of  ESTCP.  TEMTADS  2x2.  Photo 

courtesy  of  ESTCP. 
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MPV.  Photo  courtesy  of 
ESTCP. 


Classification  Planning 


•  What  parts  of  the  site  have  an  anomaly  density  that  will 
allow  classification  to  be  successful  and  at  what  cost? 

♦  Small  sites  and  densities  greater  than  1 000  per  acre  may  not  be 
appropriate. 

♦  For  sites  with  few  anomalies,  the  costs  of  remediation  must 
justify  the  extra  expense  required  for  data  collection 

■  Wetlands,  chemical  sites,  etc. 


Landfill  surprise. 
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•  What  are  the  appropriate  seed  items 
and  how  deeply  should  they  be 
buried? 

♦  Surrogates 

♦  ISOs  similar  to  known  munitions 

♦  Surrogates  of  unknown  munitions 

•  How  many  production  area  seeds? 

♦  Encountered  daily 

•  How  often  should  the  IVS  and  noise 
strip  be  surveyed? 

♦  Twice  daily 
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Blind  seed  bagged  and  buried. 
Photo  courtesy  of  ITRC 
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IVS  QC  Products  for  Detection  Sensor 


Amplitude  Reproducibility 


Position  Reproducibility 


IVS  Item  4 
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Item  4 


Implementing  Classification  -  Workflow  &  QC  Products 


13 


IVS  QC  Product  for  Classification  Sensor 


Time  (ms) 


Time  (ms) 


Time  (ms) 


■  p1  RMS  dev 

■  p2  RMS  dev 
•  p3  RMS  dev 


Time  (ms) 
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Was  the  Sensor  Correctly  Positioned 
Over  the  Target? 
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Were  All  the  Tx  and  Rx  Coils  Operating 
Within  Tolerance  Limits? 


Inversion  with  Rx3Y  (left)  and  without  (right) 
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RxO  Rxl  Rx2  Rx3  Rx4  Rx5  Rx6 
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Important  Feature  Extraction  Questions 


What  are  the  criteria  for  deciding  a  solution  has  been 
reached? 

What  are  the  criteria  for  determining  that  reliable 
features  cannot  be  extracted  for  a  particular  anomaly 
and  how  is  that  anomaly  treated? 
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Does  an  Analyst  Examine  Each  Anomaly? 
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How  Are  Multiple  Targets  Identified  and  Evaluated? 


single-source  solver 
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How  Are  Multiple  Targets  Identified  and  Evaluated? 


multiple-source  solver 
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Documentation  Required  for  the  Classifier 


•  What  features  been  identified  that  will  allow  classification 

♦  polarizability  amplitude  and  decay 

♦  depth  and  signal  decay 

•  Will  a  library  matching  or  a  statistical  based  approach  be 
used  to  classify  the  anomalies? 

•  Does  the  classifier  require  training  data,  how  many,  and 
how  are  they  chosen? 


Implementing  Classification  -  Workflow  &  QC  Products 


24 


Workflow  and  Quality  Control  Products 


12 


Implementing  Classification  on  a  Munitions  Response  Project 


December  1,  201 1 


Decay  Parameter  as  Discriminant 


Library-based  Classification 
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Classifier  Training  Data 
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Site  Characterization 
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Unknown/Unexpected  Items 


6  of  25  items  with  similar  features  at  the  Beale  Trees  site 
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Unknown/Unexpected  Items 


6  of  25  items  in  cluster  under  the  trees  at  Beale 
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QC  Seeds  On  the  Dig  List 
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How  Are  the  UXO  Classified 
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Systematic  Planning  Process 

•  Systematic  Planning  Process  is  based  on  the  scientific 
method  and  will  help  answer  the  previous  questions  in 
the  different  phases  of  work 

•  Ensures 

♦  Appropriate  amount  and  type  of  data  for  decision 

♦  Data  collected  addresses  characteristics  of  the  site 

•  Commonsense  graded  approach 

•  Promotes  communication  between  all  organizations  and 
individuals  involved  and  is  documented  in  the  UFP- 
QAPP 
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Quality  Assurance  Project  Plan  (QAPP) 


•  Why  is  it  so  important  to  accurately  document  the 
QAPP/SAP? 

♦  SAP  documents  how  QA  and  QC  are  applied  to  ensure  that  the 
results  obtained  will  satisfy  the  stated  performance  criteria. 

♦  Purpose  of  a  SAP  is  to  document  the  planned  activities  data 
collection  operations. 

♦  Provide  a  project-specific  “blueprint”  for  obtaining  the  type  and 
quality  of  environmental  data  needed  for  a  specific  decision  or 
use. 

♦  Without  a  properly  documented  plan  there  is  no  way  to 
historically  reconstruct  what  was  done  for  the  project 

•  ESTCP  is  developing  example  classification  QAPPs 
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Pole  Mountain,  WY  Case  Study 

Herb  Nelson 


Implementing  Classification  -  Case  Study 


Pole  Mountain 
Training  and 
Maneuver  Area 


•  Bisbee  Hill 
Maneuver  Area 


•  South  Firing  Area 


•  Twin  Mountain 
Impact  Area 
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•  A  variety  of  munitions  have  been  reported  as  used  at 
Pole  Mountain.  Physical  evidence  for  the  following  items 
was  discovered  during  the  Rl: 

♦  Projectiles  containing  high  explosive  (HE)  filler  (37-mm  to  155- 
mm,  and  2.95-inch); 

♦  Shrapnel  projectiles  (75-mm  and  3-inch); 

♦  37-mm  projectiles  (inert  and  unfuzed); 

♦  3-inch  Stokes  mortars  (practice,  fuzed);  and 

♦  60-mm  mortars  containing  HE  filler. 
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•  Seed  emplacement 

•  EM61-MK2  detection  survey 

♦  RTKGPS 

•  Select  anomalies  for  further  investigation 

•  Collect  cued  data  using  MetalMapper 

•  Intrusive  investigation 
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Item 

Depths 

37-mm  projectile 

15  -  30  cm 

57-mm  projectile 

20  -  35  cm 

60-mm  mortar 

30  cm 

75-mm  projectile 

20  -  40  cm 

3-in  stokes  mortar 

30  cm 

Small  ISO 

1 5  -  25  cm 
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The  Premise 


•  The  first  -25%  of  the  Pole  Mountain  site  was  surveyed  in 
the  first  field  season 

♦  Anomalies  selected  from  the  EM61  survey  data 

♦  Cued  MetalMapper  data  collected 

•  Data  analysis  is  complete  and  it  is  time  for  the  site  team 
to  make  some  decisions 

♦  Are  the  data  acceptable? 

♦  Is  the  analysis  acceptable? 

♦  Can  we  stop  digging  at  the  contractor’s  stop-dig  point? 
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PESTCP 


EM61-MK2  Data 
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•  Static  Instrument  Tests 

•  Twice  daily  IVS 

♦  One  sphere 

♦  Two  small  ISOs 

♦  One  37-mm  projectile 

♦  One  75-mm  projectile 
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Amplitude  Reproducibility  for  IVS  Item  4 
37mm  projectile 


Team  2 


Team  3 
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Position  Reproducibility  for  IVS  Item  4 
37mm  projectile 


Team  2 
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Team  3 
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Anomaly  Selection  Criterion 
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•  Anomaly  selection  based  on  37mm  projectile  at  30  cm 

♦  EM61  is  appropriate  sensor  for  this  job 

■  RMS  noise  «  0.7  mV  in  channel  2 

■  Selection  threshold  =  5.2  mV  in  channel  2 

■  938  anomalies  selected 

•  All  QC  seeds  detected  using  this  threshold 

♦  Some  just  inside  the  60-cm  halo 

•  IVS  reproducibility  within  requirements 

•  Coverage  and  measurement  density  within  requirements 
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•  Are  the  data  acceptable? 

•  Is  the  analysis  acceptable? 
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Decision  2 


Metal  Mapper  Cued  Data 
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IVS  Reproducibility 


Time  (ms) 


Time  (ms) 


Time  (ms) 


■  p1  RMS  dev 

•  p2  RMS  dev 

•  p3  RMS  dev 


Time  (ms) 
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Real-time  QC  -  Anomaly  912 
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MetalMapper  Cued  Data 


•  Cued  data  collected  over  all  anomalies 

♦  Real-time  QC 

♦  Some  re-collections 

•  IVS  reproducibility  within  specifications 

•  Are  the  data  acceptable? 
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MetalMapper  Analysis  and 
Anomaly  Ranking 
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Statistical  Analysis  -  2D  Feature  Plot 
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•  Twice  daily  IVS  +  Static  Tests 

♦  One  sphere 

♦  Two  small  ISOs 

♦  One  37-mm  projectile 

♦  One  75-mm  projectile 

•  Training  Pit 

♦  37-mm,  57-mm,  and  75-mm  projectiles  and  3-in  stokes  mortar 

♦  Two  depths  (at  least  one  with  good  SNR) 

♦  Four  orientations 
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Test  Pit  &  IVS  Results 
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Three  Training  Data  Requests 


Implementing  Classification  -  Case  Study  27 


Six  Known  Clusters 
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Small  ISO  Cluster  (1) 
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Small  ISO  Cluster  (2) 
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Unknown  Cluster  1  (6  of  39) 
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Unknown  Cluster  2 
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2D  Feature  Plot 
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Ranked  Anomaly  List  -  Final 
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•  Is  the  analysis  acceptable? 

•  Do  we  accept  the  stop-dig  point? 
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QC  The  Classification  Process 
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QC  Results 
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Year  2  Analysis  and 
Anomaly  Ranking 
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Year  2  Training  Request 


•  Do  we  need  more  training? 

•  Can  we  use  the  same  classifier  thresholds? 


Implementing  Classification  -  Case  Study  54 


Case  Study 


27 


Implementing  Classification  on  a  Munitions  Response  Project 


December  1, 2011 


0  200  400  600  800  1000  1200 

Number  of  Clutter  Items  Dug 

Implementing  Classification  -  Case  Study 


55 


Implementing  Classification  -  Case  Study  56 


Case  Study 


28 


Implementing  Classification  on  a  Munitions  Response  Project 


December  1, 2011 


•  Is  the  analysis  acceptable? 

•  Do  we  accept  the  stop-dig  point? 
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How  Did  We  Do? 
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Year  2  Full  ROC 
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Implementing  Classification  on  a 
Munitions  Response  Project 

Wrap  Up 
Vic  Wieszek 


Is  This  Ready  for  Transition? 


Demonstrated  on  a  variety  of  site  conditions 

Good  understanding  of  applicability  and  limitations 

Yes,  with  more  to  come 

Commercially  available  sensor 

Yes 

Freely  available  analysis  software 

Yes 

OSD  support 

Yes 

Contracting  challenges 

Ongoing 

Regulatory  acceptance 

Ongoing 

Train  contractor  work  force  and  DoD  PMs 

Yes,  and  ongoing 
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•  Those  responsible  for  sites  that  have  these 
characteristics 

♦  Removal  of  subsurface  munitions  is  desired 

♦  Munitions  types  -  37-mm  and  larger 

♦  Anomaly  density  up  to  about  1000  per  acre 

♦  Benign  geology 

♦  Benign  to  moderate  terrain  and  vegetation 
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Eliminate  70% 
Clutter  Digs 


Eliminate  90% 
Clutter  Digs 
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Is  This  Method  Proposed  To  Me  Really 
“Classification”? 


•  Many  approaches  will  be  called  “Classification” 

•  Hallmarks  of  success 

♦  Advanced  sensors 

♦  Principled,  physics-based  analysis 

♦  Transparent  -  all  decisions  documented  and  reviewable 

•  Things  to  watch  for 

♦  EM61  and  magnetometers  =  very  limited  classification  potential 

♦  “Black  boxes” 

♦  No  independent,  blind  testing  of  the  approach 


Results  shown  here  apply  only  to  the  systems  and  methods 
demonstrated  in  the  ESTCP  Live  Site  Program 
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•  Smaller  sensors  for  use  in 
challenging  terrain  and 
vegetation 


•  One-pass  classification  - 
eliminate  the  need  for  stationary 
cued  data 


•  Transition  from  developers  to 
production  geophysics  k  _ 

companies  H(r)  =  ^j^(r)A, 

i= 1 
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Where  Are  We  Going? 
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Take  Home  Message 


•  Accelerate  pace  of  cleanup  at  a  constant  funding  level 

♦  Completion  dates  forecast  decades  out 

♦  Reduce  risk  more  rapidly 

•  Better  understood  and  transparent  process 

♦  Higher  quality  data  collected 

♦  All  decisions  documented  and  reviewable 

♦  Minimize  operator  effect  on  quality 

♦  Ability  to  adapt  to  new  information 

•  Managing  residual  risk 

♦  Removals  are  not  perfect  under  current  practice 

♦  Some  residual  risk  will  always  remain  and  must  be  managed 

♦  Clearing  more  land  sooner  is  better 
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Advisory  Group 


OESTCP 


James  Austreng,  USACE 
Harry  Craig,  US  EPA 
Jon  Haliscak,  AFCEE 
Bryan  Harre,  NAVFAC  ESC 
Robert  Kirgan,  USAEC 
Doug  Maddox,  USEPA 
Doug  Murray,  NOSSA 
Andy  Schwartz,  USAEC 
Steve  Sterling,  CA  DTSC 
Jeff  Swanson,  Colorado  DPHE 
Jon  Ussery,  AFCEE 
Ken  Vogler,  Colorado  DPHE 
Amy  Walker,  USACE 
Ed  Walker,  CA  DTSC 
Vic  Wiesek,  ODUSD(l&E) 


•  Camp  Butner 


•  Camp  Sibert 


•  Pole  Mountain 


♦  Marty  Morgan 

♦  Raye  Livermore 


♦  Adrienne  Nunn 

♦  Jane  Francis 

♦  Rick  Grabowski 

♦  Dave  Rathke 


♦  Steve  Cobb 

♦  Tracy  Strickland 
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serdp-estcp.org 

•  Featured  Initiative  on  Classification  -  updated  as 
demonstrations  proceed 

•  Tools  and  Training 

♦  Webcasts  of  short  courses 

♦  Animation  tutorial 

♦  Summary  reports 

♦  Interim  Guidance  Document  on  Implementing  Classification 

•  Funding  Opportunities 

♦  Submit  proposals  for  upcoming  live  site  demonstrations 
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